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DETAILED ACTION 

1 . This action is in response to the interview held on March 8^^, 2007. 

Corrected Office Action 

2. As discovered during the interview, in the last office action dated 12/11/06 the 
examiner identified the wrong reference in item 4 (item 5 in the current action), which 
was a 102 rejection of claims 1, 6, 7, 9, 15, 16 and 28-45, In the last action the 
examiner erroneously cited Leiong (5,444,458). The examiner intended to cite 
Katayama (6,389,179). The error has been corrected, in this updated office action. The 
time to reply therefore has been reset. 

Claim Rejections - 35 USC § 102 

3. The following is a quotation of the appropriate paragraphs of 35 U.S.C. 102 that 
form the basis for the rejections under this section made in this Office action: 

A person shall be entitled to a patent unless - 

(e) the invention was described in (1) an application for patent, published under section 122(b), by 
another filed in the United States before the invention by the applicant for patent or (2) a patent 
granted on an application for patent by another filed in the United States before the invention by the 
applicant for patent, except that an international application filed under the treaty defined in section 
351(a) shall have the effects for purposes of this subsection of an application filed in the United States 
only if the international application designated the United States and was published under Article 21(2) 
of such treaty in the English language. 

4. Claims 1, 3-10 and 12-27 are rejected under 35 U.S.C. 102(e) as being 
anticipated by Leiong et al. (US Patent 5,444,478 A). 

Regarding claims 1 and 9, Leiong et al. discloses an image processing method 
and apparatus for correcting image distortions, comprising the steps of: 

a) an input unit inputting a plurality of partially overlapping images of a 
tangible object on an object plane, the plurality of partially overlapping images sharing a 
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common location of the tangible object and being created by capturing the tangible 
object on the tangible object plane from different directions to the tangible object plane 
(Figure 7); 

b) a correspondence detecting unit (Fig. 3, num. 200 is used in a "calibration" 
in col. 16, line 63 via the method from col. 16, line 63 to col. 17, line 13) determining a 
feature point ("seam" in col. 16, line 47 and shown in fig, IE, label: "U") of one of the 
plurality of partially overlapping images (in an "overlap zone" in col. 16, lines 49,50) 
corresponding to the common location of the tangible object (The building of figures 7A 
and 7B), and 

c) determining a matched point of one of the other partially overlapping 
images corresponding to the feature point (to create "a perfect juxtapo-sition" in col. 16, 
lines 66,67) so that a direction (Fig. 5B, direction or line "PM".) of the tangible object 
plane (Fig. 5B, label" lo") is calculated based on the feature point and the matched point 
(Note that the above mentioned perfect juxtaposition is interpreted as matching features 
of an image Si in fig. 7C to features of an image Sj in fig. 7C along a seam, L in fig. 7C 
to create a seamless image as shown in fig, 7D where no seam is visible and respective 
portions of the image such as the building and trees are matched to create "a 
uniform,, .image" in col. 17, lines 12,13.), the tangible object plane (fig. 5A, label: "M") 
being defined by a spatial orientation (of either the coordinate system (yO,Xo,Zo) of an 
image lo or the coordinate system (Y,Z,X,Zo) of a camera, "P" as shown in fig. 5A) of the 
tangible object; 
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d) a standard image setting unit (fig. 3, num. 205) selecting one ("selecting 
one" in col. 6, line 46) of the plurality of partially overlapping images (Fig, 6, num. 205) 
as a standard image (via a "reference... camera" in col. 9, line 66 that creates a "source 
image" in col. 9, line 67 which is interpreted as the claimed standard image) whose 
image distortions are to be corrected (as shown in fig. 7D relative to fig. 7C.); and 

e) a distortion correcting unit (fig. 3, num. 112) generating a distortion- 
corrected image (fig. 7D) on a projection plane (or "projected in the... plane" in col. 8, 
line 11) by projecting the standard image onto the projection plane based on the 
direction of the tangible object plane such that image distortions in the standard image 
are eliminated. 

Regarding claim 3, Leiong et al. discloses the image processing method 
according to claim 1 wherein in said selecting step, one of the plurality of partially 
overlapping images is automatically ("automatically" in col. 17, line 14) selected as the 
standard based on a direction of a straight-line pattern (fig. 1E, label: "U") contained in 
each image. 

Claims 4 and 5 are rejected the same as claim 1. Thus, argument similar to that 
presented above for claim 1 is equally applicable to claims 4 and 5. 

Claim 6 is rejected the same as claim 1 . Thus, argument similar to that 
presented above for claim 1 is equally applicable to claim 6 except for the remaining 
limitations of: 

a) selecting one of the plurality of partially overlapping images as a standard 
image (Fig. 6, num. 205) that contains a smallest amount of image distortions (via a 
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"feedback" in col. 16, line 64 to create "a target image without any faults" in col. 16, lines 
67,68. Thus, if a target image is created without any faults then the images that were 
used to create the target image also have no faults.) among the plurality of partially 
overlapping images; and 

b) combining the other partially overlapping images, which are projected (as 
shown in fig. 1A via a plurality of lines that radiate from point T".) onto an image 
surface (Fig. 1A, label: "lo") of the standard image with respect to each of the other 
partially overlapping images, so that a composite image (fig. 7D) is generated on the 
image surface so as to correct image distortions in the standard image (relative to the 
image of fig. 7C). 

Claim 7 is rejected the same as claims 1 and 9. Thus, argument similar to that 
presented above for claims 1 and 9 is equally applicable to claim 7 except for the 
limitation disclosed in Leiong et al. of: 

a) an image composition unit ("Devices capable of realiz-ing these 
operations" in col. 17, lines 5,6 such as "perfect juxtapo-sition" in col. 16, lines 66,67.) 
combining (via a "joint" in col. 9, line 42) the other partially overlapping images, which 
are projected (as shown in fig. 1 A via a plurality of lines that radiate from point "P".) onto 
an image surface (Fig. 1 A, label: "lo") of the standard image with respect to each of the 
other partially overlapping images, so that a composite image (fig. 7D) is generated on 
the image surface so as to correct image distortions in the standard image (relative to 
the image of fig. 7C). 
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Regarding clainn 8, Leiong et al. discloses the image processing apparatus 
according to claim 7, 

a) wherein said standard image setting unit (fig. 3, num. 205) is configured 
such that a user (via fig. 3, num. 2) is required to select the standard image when taking 
the original image from one of the oblique directions (Fig. 7A shows an image of a 
scene at one angle and fig. 7B shows another image of the same scene at another 
angle so that when both images are combined as shown in fig. 7C distortions are shown 
because the images of the same scene were taken at different angles.), and 

b) wherein said image processing apparatus further comprises: 

b1) a notification unit (Fig. 21, num. 2) which notifies the user that the 
standard image is currently taken (via fig. 3, num. 205). 

Regarding claim 10, Leiong et al. discloses the image processing apparatus 
according to claim 9, further comprising: 

a) a plurality of imaging units (Fig. 1G, label "P") which respectively input the 
plurality of partially overlapping images that are generated by taking the original image 
from the oblique directions. 

Claims 12-14 are rejected the same as claims 3-5, respectively. Thus, argument 
similar to that presented above for claims 3-5 is equally applicable to claims 12-14, 
respectively. 

Claim 15 is rejected the same as claims 1 and 9. Thus, argument similar to that 
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presented above for clainns 1 and 9 is equally applicable to claim 15 except for the 
additional limitation of a computer-readable storage medium as disclosed in Leiong et 
aL in fig. 3, num, 210. 

Claim 1 6 is rejected the same as claims 7 and 1 5. Thus, argument similar to that 
presented above for claims 7 and 15 is equally applicable to claim 16. 

Regarding claim 17, Leiong et al. discloses the image processing method of 
claim 1, wherein said standard image is projected with a perspective projection matrix 
operation (or " 'perspective transform'" in col. 14, line 57), 

Claims 18-22 are rejected the same as claim 17. Thus, argument similar to that 
presented above for claim 17 is equally applicable to claims 18-22. 

Regarding claim 23, Leiong discloses the image processing method of claim 17, 
wherein said perspective projection matrix is calculated based on coordinates (or 
"coordinates" in col. 14, line 61) of at least four combinations of feature points of the 
standard image and matched points corresponding thereto (via a "4 X 4 matrix" in col. 
14. line 62). 

Regarding claim 24 see "least-squares method" in col. 15, line 17. 

Regarding claim 25, Leiong et al. discloses the image processing method 
according to claim 1, wherein the tangible object is an image on a planar surface (or a 
"scene" in col. 2, line 27 that corresponds to the claimed image that has "surface area" 
in col. 2, line 27. 
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Regarding claim 27, Leiong discloses The image processing method according 
to claim 1, wherein the projection plane is parallel to the tangible object plane (Leiong's 
project plane is parallel to the imagine plane or virtual plane Co). 

5. Claims 1, 6. 7, 9, 15, 16 and 28 - 45 are rejected under 35 U.S.C. 102(e) as 
being anticipated by Katayama et al. (US Patent 6,389,179 A). 

Regarding claims 1 and 9, Leiong et al. discloses an image processing method 
and apparatus for correcting image distortions, comprising the steps of: 

a) an input unit inputting a plurality of partially overlapping images of a 
tangible object on an object plane, the plurality of partially overlapping images sharing a 
common location of the tangible object and being created by capturing the tangible 
object on the tangible object plane from different directions to the tangible object plane 
(Figure 6, item 100); 

b) a correspondence detecting unit determining a feature point of one of the 
plurality of partially overlapping images corresponding to the common location of the 
tangible object (Figure 10, item 172b), and 

c) determining a matched point of one of the other partially overlapping 
images corresponding to the feature point so that a direction of the tangible object plane 
is calculated based on the feature point and the matched point, the tangible object plane 
being defined by a spatial orientation of the tangible object (Figure 10, item 172b); 
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d) a standard inriage setting unit selecting one of the plurality of partially 
overlapping images as a standard image whose image distortions are to be corrected 
(col. 10, lines 49 -57); and 

e) a distortion correcting unit generating a distortion-corrected image on a 
projection plane by projecting the standard image onto the projection plane based on 
the direction of the tangible object plane such that image distortions in the standard 
image are eliminated (Figures 7 and 8). 

Claim 6 is rejected the same as claim 1. Thus, argument similar to that 
presented above for claim 1 is equally applicable to claim 6 except for the remaining 
limitations of: 

a) selecting one of the plurality of partially overlapping images as a standard 
image that contains a smallest amount of image distortions among the plurality of 
partially overlapping images (col. 10, lines 49 - 57); and 

b) combining the other partially overlapping images, which are projected onto 
an image surface of the standard image with respect to each of the other partially 
overlapping images, so that a composite image is generated on the image surface so as 
to correct image distortions in the standard image (Figures 7 and 8). 

Claim 7 is rejected the same as claims 1 and.9. Thus, argument sinnilar to that 
presented above for claims 1 and 9 is equally applicable to claim 7 except for the 
limitation of: 

a) an image composition unit combining the other partially overlapping images, 
which are projected onto an image surface of the standard image with respect to each 
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of the other partially overlapping innages, so that a composite image is generated on the 
image surface so as to correct image distortions in the standard image (Figures 7 and 
8). 

Claim 15 is rejected the same as claims 1 and 9. Thus, argument similar to that 
presented above for claims 1 and 9 is equally applicable to claim 15.. 

Claim 16 is rejected the same as claims 7 and 15. Thus, argument similar to that 
presented above for claims 7 and 15 is equally applicable to claim 16. 

In regards to claim 28, the image processing method according to claim 1, 
wherein the partially overlapping images are created by capturing the tangible object on 
the object plane using a same imaging unit (Figure 6). 

In regards to claim 29, the image processing method according to claim 1, 
wherein the plurality of partially overlapping images are created by capturing the 
tangible object on the object plane at different time instants using a same image unit 
(Figure 6). 

In regards to claim 30, the image processing method according to claim 1, 
wherein the plurality of partially overlapping images are created by moving a same 
imaging unit to capture the tangible object on the object plane from different directions 
to the tangible object plane (Figure 6). 

In regards to claims 31, 34, 37, 40, and 43, claims 31, 34, 37, 40, and 43 are 
rejected for the same reasons as claim 28. The argument analogous to that presented 
above for claim 28 is applicable to claims 31 , 34, 37, 40, and 43. 
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In regards to claims 32, 35, 38, 41, and 44, claims 32, 35, 38, 41, and 44 are 
rejected for the same reasons as claim 29. The argument analogous to that presented 
above for claim 29 is applicable to claims 32, 35, 38, 41 , and 44. 

In regards to claims 33, 36, 39, 42, and 45, claims 33, 36, 39, 42, and 45 are 
rejected for the same reasons as claim 30. The argument analogous to that presented 
above for claim 30 is applicable to claims 33, 36, 39, 42, and 45. 

Claim Rejections - 35 USC §103 

6. The following is a quotation of 35 U.S.C. 103(a) which forms the basis for all 
obviousness rejections set forth in this Office action: 

(a) A patent may not be obtained though the invention is not identically disclosed or described as set 
forth in section 102 of this title, if the differences between the subject matter sought to be patented and 
the prior art are such that the subject matter as a whole would have been obvious at the time the 
invention was made to a person having ordinary skill in the art to which said subject matter pertains. 
Patentability shall not be negatived by the manner in which the invention was made. 

7. Claims 2 and 11 are rejected under 35 U.S.C. 103(a) as being unpatentable over 
Leiong et al. (US Patent 5,444.478 A) in view of Lee (US Patent 6,507,366 B1 ). 

Regarding claim 2, Leiong et al. teaches the image processing method according 
to claim 1 wherein in said selecting step, one of the plurality of partially overlapping 
images is automatically (or "automatically" in col. 17, line 14) selected (via fig. 3, num. 2 
which can be an automatic process that selects the standard image via fig. 3, num. 205.) 
as the standard image. 

Leiong et al. does not teach the remaining limitation of selecting the standard 
image based on a ratio of an area of an object region to an entire area of each image, 
but does teach an "automatic function" in coL 4, line 61 for "monitor[ing]" in col. 4, line 
40. Thus, Leiong et al. suggests that an automatic process can be used for monitoring 
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automatically, but does not provide enough details on the automatic aspect of 
monitoring. Thus, one of ordinary skill in the art will be motivated to find an automatic 
process of monitoring so that a person is freed from monitoring a scene and so that the 
person can perform other duties. 

Lee teaches "monitoring" in col. 1, line 12 and "automatically tracking" in the 
abstract as taught in Leiong et al. and the remainirig limitation of claim 2: 

a) selecting (via fig. 3A,num. 360) based on a ratio of an area of an object 
region (or "region ratio" in col. 9, line 1) to an entire area (or "effective region" in col. 9, 
line 3) of each image. 

It would have been obvious at the time the invention was made to one of ordinary 
skill in the art to modify Leiong et al.'s teaching of selecting the standard image and 
automatic monitoring with Lee's teaching of selecting or automatic tracking via fig. 
3A,num. 360 so that Leiong et al's selecting the standard image, either one of Ij shown 
twice as shown in fig. IE of Leiong et al. can be automatically tracked or monitored 
freeing a person from monitoring to perform other duties. 

Claim 1 1 is rejected the same as claim 2. Thus, argument similar to that 
presented above for claim 2 is equally applicable to claim 1 1 . 

8. Claim 26 is rejected under 35 U.S.C. 103(a) as being unpatentable over Leiong 
et al. (US Patent 5,444,478 A) in view of Honma (US Patent 6,304,313 B1). 

Regarding claim 26, Leiong et al. does not teach the limitations of claim 26 of a 
smallest inclination angle of a viewing direction of the oblique imaging, but 



Application/Control Number: 09/940,432 Page 13 

Art Unit: 2624 

does teach that "All the means for correct-ing the distortions of the objectives may be 
known to those skilled in the art." in col. 14, lines 4,5. Thus, Leiong et al. suggests to 
one of ordinary skill in the art that any means can be used to correct distortions of an 
objective. 

Honma teaches a means of correcting a distortion using "perspective correction" 
in col. 13, line 20 of an objective or "optical lens" in col. 13, line 20 and the remaining 
limitation of: 

a smallest inclination angle (or "correctly face[d]" in col. 11, line 2) of a viewing 
direction (of a "camera" in col. 1 1 , line 1 .). 

It would have been obvious at the time the invention was made to one of ordinary 
skill in the art to modify Leiong et aL's teaching of a means to correct distortions of an 
objective with Honma's teaching of perspective correction, because Honma's teaching 
provides a "[simple] lens" in col. 13, line 21 with "less cost than conventional devices." in 
col. 13, line 25. 

In addition, Honma's teaching of when correctly facing a camera, "perspec-tive 
correction is unnecessary" in col. 11, lines 3,4. Thus, extra time is saved by bypassing 
the step of perspective correction when the camera is correctly faced. 

Response to Arguments 

9. Applicant's arguments filed 09/28/06 have been fully considered but they are not 
persuasive. 

10. Before addressing the applicant's arguments the examiner would first like to 
address the core issue. In the last office action dated 06/28/06, item 7, the examiner 
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stated that "Leiong never calculates an object plane at the tangible object plane, but 
merely uses a virtual plane lo of a virtual camera view Co It is believed that this 
statement is the basis for the applicant's amendments. However, the current language 
in the claims does not call for calculating an object plane at the tangible object plane. If 
this language was included in the claims Leiong would be overcome. 

11. The applicant's first argument focuses on the tangible object plane. This term is 
not defined in the specification and therefore can be considered to include just about 
anything that can be defined as a plane. 

12. In response the applicant's arguments dealing with the matched point, please 
see the rejection provided above. Leiong does find the matched point and determine 
the direction. 

13. In response the applicant's arguments dealing with "point" M. Point M can be 
considered a plane, that plane is simply defined by the orientation to other planes. 

Conclusion 

14. Applicant's amendment necessitated the new ground(s) of rejection presented in 
this Office action. Accordingly, THIS ACTION IS MADE FINAL. See MPEP 
§ 706.07(a). Applicant is reminded of the extension of time policy as set forth in 37 
CFR 1, 136(a). 

A shortened statutory period for reply to this final action is set to expire THREE 
MONTHS from the mailing date of this action. In the event a first reply is filed within 
TWO MONTHS of the mailing date of this final action and the advisory action is not 
mailed until after the end of the THREE-MONTH shortened statutory period, then the 
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shortened statutory period will expire on the date the advisory action is nnailed, and any 
extension fee pursuant to 37 CFR 1.136(a) will be calculated from the mailing date of 
the advisory action. In no event, however, will the statutory period for reply expire later 
than SIX MONTHS from the date of this final action. 

Any inquiry concerning this communication or earlier communications from the 
examiner should be directed to Christopher L. Lavin whose telephone number is 571- 

272- 7392. The examiner can normally be reached on M - F (8:30 - 5:00). 

If attempts to reach the examiner by telephone are unsuccessful, the examiner's 
supervisor, Bhavesh M. Mehta can be reached on (571) 272-7453. The fax phone 
number for the organization where this application or proceeding is assigned is 571- 

273- 8300. 

Information regarding the status of an application may be obtained from the 
Patent Application Information Retrieval (PAIR) system. Status information for 
published applications may be obtained from either Private PAIR or Public PAIR. 
Status information for unpublished applications is available through Private PAIR only. 
For more information about the PAIR system, see http://pair-direct.uspto.gov. Should 
you have questions on access to the Private PAIR system, contact the Electronic 
Business Center (EBC) at 866-217-9197 (toll-free). If you would like assistance from a 
USPTO Customer Service Representative or access to the automated information 
system, call 800-786-9199 (IN USA OR CANADA) or 571-272-1000. 




Christopher Lavin 




